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Abstract 



A software tool designed to support collaborative discourse between 
multiple participants is described. The Multimedia Forum Kiosk 
supports students 1 knowledge building as they read issues, reflect on 
comments, and develop a point of view. The system provides two 
representations for fostering better comprehension of discourse and " 
reflection through classification of comments into an argument map. 
Multimedia elements (digitized video, audio, images) are posted to 
stimulate discussion. Its uses as an instructional tool and assessment 
tool, as well as a methodology for analyses of comments, are described. 
Guidelines for successful implementation in instructional settings 
based on classroom experiences are provided. 

Introduction 

This paper reports on the design and testing of an electronic multimedia bulletin 
board called the Multimedia Forum Kiosk (MFK)* Through reading about issues, 
reflecting on comments, and posting opinions, members of a community can use 
MFK to exchange thoughts, negotiate meanings, acquire knowledge, and provide 
data for assessment. Unlike previous technology-based tools for collaborative 
discourse [Scardamalia et al, 1989; Scardamalia & Bereiter, 1991; Kunz & Rittel, 
1972] or earlier bulletin board models [Hsi, 1992; Kerns, 1991], MFK provides 
explicit rhetorical representations that support user comprehension and 
user /group interactions. Comments are accessed through face icons of 
conversation participants. Argument trees diagram conversation structure as it 
develops. Multimedia clips (digitized video and audio) are used to set the 
context for topics and serve as discussion starters. The MFK not only serves as a 
catalyst for discussion and learning, but also provides a vehicle appropriate for 
educational assessment. 
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We have examined how MFK supports group interaction through asynchronous 
discourse as well as how it documents community-wide views. Communication 
models have been described as one-to-one, one-to-many, and many-to-many. The 
Kiosk is a tool that provides a many-to-many communication, but allows a single 
user to participate at his or her own pace* In the following sections, we outline 
our theoretical perspective, explain the design o* MFK system, describe our 
methodology of examining Kiosk comments, and then report on our experiences 
in using Kiosk for collaboration and assessment. 

Learning Through Collaborative Discourse and Conversation 

We postulate several ways in which social context and collaborative discourse 
can improve individual learning. People construct their understanding through 
everyday conversation both in and out of the classroom. In a typical classroom 
discussion, each individual is responsible for making sense of a discussion and 
making a contribution. Instructors, in turn, are responsible for facilitating the 
discussion, and monitoring the comprehension and participation of the group. 
However, discussion is difficult for those who don't interrupt or fail to get the 
attention of the instructor due to social norms and expectations. Also, discussion 
participants who have different levels of background knowledge on a subject are 
less likely to participate in the conversation. Learners at first may require some 
scaffolding before integrating community views and knowledge into their own 
thinking [Brown, Collins and Duguid, 1989, Newman, 1989]. Eventually, 
students should be encouraged to formulate their own goals, ask their own 
questions, direct their own inquiry, and do their own monitoring of 
comprehension, yet be able to contribute at their own pace. An activity such as 
this in which students learn by contributing to the knowledge of a group is 
termed knowledge building. [Scardamalia & Bereiter, 1991]. Our objective was 
to design a system for which this type of learning is facilitated, so that people 
can ask their "knowledge-building" questions in a non-threatening environment. 

On the individual level, reflection is an important activity to foster this 
knowledge building. Students should actively think about issues, inspect 
individual arguments, evaluate different points of view, and organize their own 
view before responding constructively. Our system is designed to facilitate these 
reflective activities as well. 
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Design of the Multimedia Forum Kiosk 

The Kiosk organizes discussion around several interesting topics that are 
suggested by the instructor or by students in the form of a question (see Figure 
1). Although not necessary to promote constructive discourse, multimedia can be 
used to illustrate various aspects of the topic and to stimulate discussion. For 
instance, digitized video dips can show a curious lab experiment, a student 
expressing concern about a difficult to understand concept, or an image of a 
piece of instruction from class. F6r example, "What parts of the Bike Lab were 
most valuable to you?" is a question that might be posted by the instructor and 
stimulus materials could include a digitized video clip of students taking a 
bicycle apart during a laboratory session. To encourage more engaging 
comments, the author of the topic provides an audio or text overview that better 
frames the question. Participants in a Kiosk conversation access an author's 
overview by clicking on his or her face. 

To participate in the community discussion, people need a sense of the other 
individuals in the community and their views. The Opinion Area screen was 
specifically designed to capture community-wide views on a specific topic (see 
figure 2). Users can hear an author's overview, watch digitized video clips, read 
the comments of others, or post comments of their own. Each comment is an 
overview of an individual's perspective on the issue. Comments are represented 
by a digitized icon of the face of the comment's contributor. This makes the 
contributor's identity salient to other users, and also encourages the contributor 
to self-monitor since the comment is i nmistakably theirs. However, topics that 
address students 1 concerns or topics sensitive in nature may allow anonymous 
participation. 

Discussion also entails negotiating, arguing, and taking sides. To accommodate a 
deeper discussion of issues, the Discussion Area screen was developed to 
complement the Opinion Area. The Discussion Area captures the relational 
structure of arguments and evolution of the conversation. Various threads of the 
discussion are represented so that users can trace an argument, line of reasoning, 
or origin of a thought (see figure 3). Information in the Discussion Area is 
organized like a discourse, so that people may understand it in ways analogous 
to understanding a conversation. 
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Before adding a comment in the Discussion Area, users are asked to classify their 
comment as an elaboration, alternative, critique, paraphrase, or question using 
semantic labels "and," "or," "but", "i.e.", and "?" (see figure 4). These labels serve to 
structure interactions, and scaffold users in following and participating in a 
discourse. The activity of categorizing and placing of comments in an existing 
map of arguments seems to foster reflection and self-monitoring, as indicated by 
a pilot study [Hoadley, Hsi & Schwarz, 1993). Users were found to explore, 
explain, and reflect before participating in a discussion. Users cyclically 
comprehended others' viewpoints, reflected on those opinions , and then 
expressed their own. [Hoadley & Hsi, 1993]. 

Thus, the Kiosk is designed to help individuals learn by interpreting the words of 
others and reflecting on their own experiences. The Kiosk encourages 
metacognitive activities such as reflection, self-critique, and monitoring one's 
own understanding. These activities can aid in knowledge integration and 
comprehension [Inagaki, 1981]. Also, by interacting with a group, users 
encounter different levels of expertise. This contributes to scaffolding of 
knowledge integration [Linn & Burbules, 1992]. The types of learning 
conversations Pea suggests for promoting conceptual change may also be 
supported by the MFK [Pea, 1992] . 

The system is currently implemented in HyperCard™ with Quicktime™. The 
HyperCard authoring tool supported on the Apple platform allowed for rapid 
prototyping, use of visual representations of discourse participants, and easy 
linking of multimedia elements. (We postulate that this will also facilitate future 
distribution and use of our tool.) 

Uses of the Multimedia Forum Kiosk 

The Multimedia Forum Kiosk is designed to be flexible for both casual and 
formal learning environments. Because of its ability to foster communication and 
leave behind a comprehensible, structured record of the discussion, the system is 
ideal for use in many situations in which people communicate. Classrooms, 
museums, offices, laboratories: any of these settings could potentially benefit 
from using the Kiosk for brainstorming, consensus reaching, or soliciting 
opinions. In education, students, teachers, and researchers can make use of 
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feedback collected by the Kiosk. For research purposes, all the comments and 
discussions are logged and time stamped electronically. This method of 
gathering data and instructional feedback provides an important bridge between 
the coarse-grained analysis of surveys and the fine-grained analysis of videotape. 
Surveys can collect general community views but not interactions between 
individuals, while video requires lengthy analysis and can overwhelm the 
researcher with details. 

Other situations where one might find the Multimedia Forum Kiosk or systems 
like it are primary or secondary school classrooms. Here, the system could 
operate quietly in the background, collecting comments on issues related to 
school or home-life. The time-independent nature of the system would allow all 
students access to the discussion. This provides a friendly opening for students 
too timid to insert their comments in boisterous verbal class discussions. It also 
permits students with linguistic or developmental disadvantages to have extra 
time. 

We now describe the contexts in which the MFK has been and is currently being 
used. The first use of the Kiosk was within our own community, for sharing 
views on current educational issues, discussing research videos, and talking 
about issues specific to our department. It was installed in the department 
lounge, where people often came to read, get tea or use the microwave. The 
topics included two discussions of general interest (about privatization of public 
schools and uses of multimedia in education), video clips from two research 
groups, two discussions directly related to the department (on allocation of 
computers within the department and on whether the department was a 
supportive community), and lastly a literally experimental topic— the video 
shown was of a short science experiment involving an egg being pushed into a 
flask. This video of the egg experiment was a model of how Kiosk could be used 
to promote scientific reasoning. For example, this Kiosk topic prompted these 
reflections about heat, thermal expansion, and air pressure. 

"I think it's because the flame experts the flask from the heat of the flame." 

"... it doesn't look like it fell through the neck due to the expansion of the flask due to heat. 
Slow-motion reveals the egg's elongation, which would not result for the neck's expansion. 
Also, the heat is probably not sufficient for that..," 
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"Well, the egg gets into the flask because the air pressure in the flask is far less than that 
outside the flask. The pressure inside is decrease by the flama, which uses up all the available 
oxygen, creating a mini-vacuum.*' 

The burning paper fixes oxygen (i.e., it reacts with carbon, etc in the paper), reducing the 
number of gaseous molecules in the bottle, hence creating a relative vacuum that sucks in the 

egg-" 

Our initial success in supporting a variety of discussion topics led us to try the 
MFK for a new purpose. The rich and interesting record produced by the Kiosk 
suggested using the Kiosk to document both the comments of individuals who 
are in the process of understanding and engaging in knowledge building, and to 
document community views or "classroom Gestalt". We have begun using the 
system as an assessment tool to collect course feedback and assess effects of 
curricular changes that might not show up on standardized tests or grades. 

Evaluation and feedback on instruction 

As part of a national effort to improve engineering education through innovative 
applications of technology [Ingraffea et al, 1990], we have tailored and used MFK 
as a method of assessing the impact of changes to new curricula by capturing 
views and reactions of students and faculty. MFK provides a way to document 
and evaluate activities for courses where feedback is often difficult to collect: in 
design studios, computer learning centers, and unscheduled laboratories. In our 
research experiment, MFK is being used (a) to monitor students' understanding 
of concepts and course materials (b) to contrast views of individuals with 
different expectations, norms, and experiences, (c) to foster dialogue and 
discussion about courseware innovations, (d) to document changes in student 
and instructor views, and (e) to examine changes over time. We are especially 
interested in retention issues, in reactions to technology-based curricular 
innovations, and in improvement of attitudes towards women and minorities in 
engineering. By using a Kiosk, we can both document knowledge building by 
capturing on-going discourse, and perform assessment by collecting student and 
faculty opinions. 

Students who want to voice opinions about new curricular materials, the 
learning atmosphere, or instructional delivery can interact with the Kiosk during 
the semester either voluntarily or as an assigned independent activity. Kiosk 
comments received during the course enable instructors to modify the course 
and better tailor it to students' needs. An example of this is provided by a Kiosk 
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discussion in an engineering design course in which students advocated more 
hands-on experiences. The questioit presented was ,f What are the best ways to 
teach design? 0 . 

"1 tend to think that any method that delves fairly deeply into the design process would be 
beneficial. Any type of hands-on experience would, therefore, be helpful to really learn all of the 
problems associated with design...** 

In my experience, design has to be taught through a variety of techniques. 1 foi one would rather learn 
through activities and interactions with other members of the class rather than the traditional lecture 
format of academics. 1 * 

"In my undergraduate Aerospace Engineering studies we spent 10 weeks studying stuff like: 
Kutta-Jakowski lifting theory, aerodynamics, etc At the end of my career I realized that I did not even 
know what an airfoil looked like or how it was fabricated. I strongly advocate hands-on education." 

I've done both case studies and actual design in industry. I personally learn better by experience than 
by classroom learning and reading. However, I still think that classroom learning and reading can 
always help, never hinder, learning design. In fact, it can give you ideas on how to improve your - 
designing skills.** 

**I think it is extremely useful to provide hands-on projects. A review of the history of design would also 
prove very effective. Analyzing the state of current technology, even if in a vary basic manner, I have 
always found to be helpful.** 

Thus, the class had a consensus that the course should involve as much hands-on 
experience as possible, which enabled the instructor to respond to these needs. 
As a result of this Kiosk feedback, the instructor changed her curriculum and 
included a hands-on design project. 

Methodology for Kiosk Data Analyses 

Comments collected by the Kiosk can be used in mid-semester assessment for 
instructors to monitor feedback continuously, or as end-of-course evaluation for 
education researchers to analyze the pooled data. For the purposes of evaluating 
auricular reform objectives of the Synthesis Coalition, several methods of data 
analysis were employed that explored usage, individual comments, and issues 
raised by participants. 

Analyses of overall usag e 

For each topic, the total number of comments, the number of individual named 
participants, and the frequency of participant by a single individual were 
tabulated. (Anonymous comments were included in the overall comment count, 
but not in the participant total.) This provided researchers a quantitative measure 
of Kiosk usage, and an indirect measure for the success of a Kiosk topic. 
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Analyses by planned themes 

To ascertain students' concerns and interests, and to find out v/hether the 
curricular innovations and new instructional materials were working, topics 
were designed around four themes relevant to the instructional goals of the 
Synthesis Coalition: industry relevance, quality ;f pedagogy (learning 
experience), class atmosphere (especially as related to equity and retention 
issues), and open-ended design- We were interested in v/hether or not students 
felt prepared for engineering practice, the nature of their learning experiences 
(interactive, hands-on, teamwork, high-tech classrooms), and whether or not the 
course provided a supportive environment for learning* The themes were not 
designed to be exhaustive, but to provide a useful framework to help design 
relevant Kiosk topics and analyze the data- 

A coding scheme based on the themes was used to categorize comments (Table 
1). For each coding category, we provide a sample comment that is 
representative of the category. Many comments fit into multiple categories. 



Table 1 



Coding Categories of Kiosk Comments by Themes 



Coding 



Engineering Theme 



Description/Example 



Major Categories 



A Industry Relevance 



"The measure to become a better 
engineer is relative to the amount 
of work that one obtains through 
work experiencr Work 
experience does not have to be 
limited to technical work in 
industry. For example during my 
undergraduate studies I 
participated in an S+A.E 
competition and personally 
through participation in the 
competition as well as organizing 
the event I became a better 
engineer in area the technical 
areas as well as the administrative 
field." 
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B 


Learning experiences 

< 


"Experience is very important 
since you will meet a lot of things 
you haven't expected before And 
knowledge is also very important, 
especially nowadays. Information 
plays an important role on 
success of design. Knowing why 
others are successful or 
unsuccessful will benefit us a lot. 
I am very impressed by case 
studies of this class and 
multimedia is a very effective 
way for us to study. Some 
readings are also good, but not as 
impressing (sic) as this method." 




C 


Classroom environment 


"Being able to learn and explore 
on my own without too much 
pressure *nade learning much 
more comfortable." 




D 


Design 


"One of the best experiences I 
have had as a designer was when 
I was entrusted the overall 
responsibility for the design of a 
mechanical system. Prior to that 
my focus was narrow, and I 
didn't appreciate the complex 
interrelationships between 
design parameters. But once I 
was forced to "see the big 
picture", I realized I could get a 
good design, only by synthesizing 
the efforts of people from 
different disciplines." 


Table 2 

Sample Subcategory Coding Scheme 

Coding Engineering Theme DescriptionlExample 


Content Subcategories 


bl 


teamwork 


"Many people from different 
backgrounds help make a group 
successful. From the different 
case studies we have seen, the use 
of multifunctional design groups 
are emphasized. The design 
groups consist of people from 
various backgrounds (design 
engineer, manufacturing, 
management, etc.)." 
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b2 


case studies 


"I think that all of the case studies 
have helped me sctnewhat to 
become a better engineer. Having 
no industrial experience, I at least 
have more of a taste of what 
design is. The case studies 
emphasize parts of design which 
are not mentioned in most 
classes...like teamwork, 
combining of design/ 
manufacturing/ marketing/ 
customer involvement/ etc., and 
management strategies. I think 
that a good engineer has to 
consider all of these factors.** 


- 


b3 


hands-on 


I agree that hands-on experience 
and the real issues such as DFM, 
DFA, and DFS (design for 
serviceability) can not be 
replaced. If you read about how 
to change automobile breaks in a 
manual, you may remember it for 
a week. If you actually doit 
however, You will remember it 
much longer. 




M 


analysis vs. theory vs. 

cvrn fli oci G 
2> V 1 1 Li lCol£> 


*1 found that working in industry, 
having a good theoretical 
understanding was only good 
when coupled with a strong 
hands-on background. (Theory 
and background are important 
and a good engineer needs to be 
well balanced.)* 1 




b5 


multiple learning styles 


Tm not sure if this question is for 
ME298P or in general, but the 
classroom environment is 
generally supportive for those 
who learn well from the lecture 
style of teaching. This class is the 
most progressive that I've seen at 
Cal because it uses many teaching 
styles and techniques. This 
variety drives an environment 
that supports learning from the 
many students who don't learn 
best from lecture.** 



Analyses of emergent issues 

One method of looking at Kiosk data is to identify key issues that are raised for 
each Kiosk topic. In many cases, the class largely agreed on which points were 
important. We examined the comments and prepared summaries of issues 
raised by participants, and tabulated the number of students who discussed a 
particular issue. In some cases, we also tabulated "pro" and "con" votes for the 
issue, if there were clear alternatives. For instance, one student raised an issue 
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about whether the professor encouraged students to participate in the class 
discussion* After a barrage of comments, it was dear that most students felt 
comfortable participating in class, but that language skills were a barrier for a 
minority of the students- Because the professor had access to a summary of this 
information, she was able to adjust her teaching appropriately* 

Guidelines for Kiosk Success 

Work to date from using Kiosk in six different engineering courses demonstrates 
that the success of the Kiosk approach in classrooms depends on specific aspects 
of the situation. In particular, our experiences suggest the following guidelines 
for successful Kiosk installations. 

(1) Introduce MFK as part of the course - To familiarize students with MFK, the 
instructor should allocate class time to demonstrate and explain the MFK. 
Faculty members must be responsive to student comments. Students are more 
interested if they feel their feedback will have consequences for the course and 
effect changes. 

(2) Enforce mandatory participation - Based on our experiences with Kiosk, we 
believe that assessment should be integrated with the curricula and be part of the 
planned instruction. We found better quality and quantity of comments if the 
MFK was included in the course syllabus. Ideally, course assignments are given 
that require mandatory participation in the Opinion Area, and students are 
rewarded for their participation, reflection, and suggestion of topics for future 
discussion. 

(3) Develop continuous monitoring and feedback on multiple topics - To support on- 
going discussion, Kiosk topics that change on a regular basis (for example, at 3-4 
week intervals) keep students interested in the evolving conversation. To reward 
students for reflection and participation in collaborative discourse, instructors 
should give summaries of comments back to students within a week's time. 

(4) Faculty participation and commitment - To have a truly successful Kiosk experience, 
instructors must be committed to evaluation and be interested in having their 
instruction assessed. To promote faculty participation, faculty should initiate the 
topics to be discussed and authored topics. We found Kiosk discourse successful 
when faculty showed enthusiasm for using Kiosk, adding to the conversation, and 
responding to students' comments. 

(5) Ensure access - The system must be placed in a public location that is 
frequented by students. Students must also have time to read and make 
comments. Open course laboratories proved most successful, while restricted- 
hours laboratories or lecture halls were unacceptable. Current plans include 
developing networked versions of the system that could be used from a dorm 
room or computer center. 
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In general, our perspective on what the MFK represents has evolved. Initially, 
the software itself and the interface were the focus. Now we look upon the MFK 
as a collection of stimulus materials, as a general approach to data collection, and 
as an aspect of the curriculum itself. Fostering and maintaining a quality 
interaction is a far more complex task than building software to support such an 
interaction. It requires the participation and enthusiasm of the participants and 
instructors as much as the developers. 

Future Work 

The strength of collaborative discourse lies in supporting multiple users, 
allowing remote access to the on-^oing conversation, and empowering 
instructors to create their own Kiosk discussions. Future plans include the 
development of an instructor's tool kit that will enable instructors to design 
topics, access a collection of stimulus materials, and select the participants in 
group discussions. To support sharing of Kiosk materials and knowledge 
building, plans are underway to design a networked version of Kiosk that can be 
linked to the World Wide Web. Before the Kiosk design can be improved, 
education researchers need to better understand the nature of social cognition 
and collaborative discourse. Research that includes discourse analyses of Kiosk 
comments is also planned. 

Conclusions 

The Multimedia Forum Kiosk has shown potential as a medium for collaborative 
discourse and reflection. Our experiences using the system in classrooms has 
shown that useful discussion can take place among many users, including those 
with limited English skills or low assertiveness in the instructional setting. By 
thinking about the questions and stimulus materials, the participants reflected on 
their own learning processes and those of others. A record of this is then 
available for researchers, instructors, and other students alike. In this way, the 
Kiosk acts as a powerful platform for productive discussion and interactions. 

Kiosk interactions depend on the social commitment of the participants involved. 
Not surprisingly, Kiosk success for assessment depends on the involvement and 
enthusiasm of instructors. The single most important factor in using electronic 
communication media is the strength of the community in which it is used. 
Accordingly, individuals need to want to participate in collaborative discourse. 
Documenting community views requires active and thoughtful participation by 
the whole community. However, when used in conjunction with vibrant 
community, structured electronic discourse can enhance the possibilities for such 
useful interactive communication and knowledge building. 
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